International Conference on Machine Learning

Kernel methods combined with large-margin learning algorithms such as SVMs have been used successfully to tackle a variety of learning tasks since their introduction in the early 90s. However, in the standard framework of these methods, the choice of an appropriate kernel is left to the user and a poor selection may lead to sub-optimal performance. Instead, sample points can be used to select a kernel function suitable for the task out of a family of kernels fixed by the user. While this is an appealing idea supported by some recent theoretical guarantees, in experiments, it has proven surprisingly difficult to consistently and significantly outperform simple fixed combination schemes of kernels. This talk will survey different methods and algorithms for learning kernels and will present novel results that tend to suggest that significant performance improvements can be obtained with a large number of kernels. (Includes joint work with Mehryar Mohri and Afshin Rostamizadeh.) Biography: Corinna Cortes is the Head of Google Research, NY, where she is working on a broad range of theoretical and applied large-scale machine learning problems. Prior to Google, Corinna spent more than ten years at AT&T Labs Research, formerly AT&T Bell Labs, where she held a distinguished research position. Corinna’s research work is well-known in particular for her contributions to the theoretical foundations of support vector machines (SVMs) and her work on data-mining in very large data sets for which she was awarded the AT&T Science and Technology Medal in the year 2000. Corinna received her MS degree in Physics from the Niels Bohr Institute in Copenhagen and joined AT&T Bell Labs as a researcher in 1989. She received her Ph.D. in computer science from the University of Rochester in 1993. Corinna is also a competitive runner, placing third in the More Marathon in New York City in 2005, and a mother of two. How do infants bootstrap into spoken language?: Models and challenges Emmanuel Dupoux Ecole Normale Superieure, Ecole des Hautes Etudes en Sciences Sociales, Centre National de la Recherche Scientifique, France Abstract: Human infants learn spontaneously and effortlessly the language(s) spoken in their environments, despite the extraordinary complexity of the task. Here, I will present an overview of the early phases of language acquisition and focus on one area where a modeling approach is currently being conducted using tools of signal processing and automatic speech recognition: the unsupervized acquisition of phonetic categories. During their first year of life, infants construct a detailed representation of the phonemes of their native language and lose the ability to distinguish nonnative phonemic contrasts. Unsupervised statistical clustering is not sufficient; it does not converge on the inventory of phonemes, but rather on contextual allophonic units or subunits. I present an information-theoretic algorithm that groups together allophonic variants based on three sources of information that Can be acquired independently: the statistical distribution of their contexts, the phonetic plausibility of the grouping, and the existence of lexical minimal pairs. This algorithm is tested on several natural speech corpora. We find that these three sources of information are probably not language specific. What is presumably unique to language is the way in which they are combined to optimize the emergence of linguistic categories. Human infants learn spontaneously and effortlessly the language(s) spoken in their environments, despite the extraordinary complexity of the task. Here, I will present an overview of the early phases of language acquisition and focus on one area where a modeling approach is currently being conducted using tools of signal processing and automatic speech recognition: the unsupervized acquisition of phonetic categories. During their first year of life, infants construct a detailed representation of the phonemes of their native language and lose the ability to distinguish nonnative phonemic contrasts. Unsupervised statistical clustering is not sufficient; it does not converge on the inventory of phonemes, but rather on contextual allophonic units or subunits. I present an information-theoretic algorithm that groups together allophonic variants based on three sources of information that Can be acquired independently: the statistical distribution of their contexts, the phonetic plausibility of the grouping, and the existence of lexical minimal pairs. This algorithm is tested on several natural speech corpora. We find that these three sources of information are probably not language specific. What is presumably unique to language is the way in which they are combined to optimize the emergence of linguistic categories. Biography: Emmanuel Dupoux is the director of the Laboratoire de Sciences Cognitives et Psycholinguistique in Paris. He conducts research on the early phases of language and social acquisition in human infants, using a mix of behavioral and brain-imaging techniques as well as computational modeling. He teaches at the Ecole des Hautes Etudes en Sciences Sociales where he has set up an interdisciplinary graduate program in Cognitive Science. Drifting games, boosting and online learning Yoav Freund University of California, San Diego, U.S.A. Abstract: Drifting games is a mathematical framework for modeling learning problems. In this talk I will present the framework and show how it is used to derive a new boosting algorithm called Robustboost and a new online prediction algorithm called NormalHedge. I will present two sets of experiments using these algorithms on synthetic and real world data. The first experiments demonstrate that Robustboost outperforms Adaboost and Logitboost when there are many outliers in the training data. The second set of experiments demonstrate that a tracking algorithm based on NormalHedge is more robust against noise than particle filters. Drifting games is a mathematical framework for modeling learning problems. In this talk I will present the framework and show how it is used to derive a new boosting algorithm called Robustboost and a new online prediction algorithm called NormalHedge. I will present two sets of experiments using these algorithms on synthetic and real world data. The first experiments demonstrate that Robustboost outperforms Adaboost and Logitboost when there are many outliers in the training data. The second set of experiments demonstrate that a tracking algorithm based on NormalHedge is more robust against noise than particle filters. Biography: Yoav Freund is a professor of Computer Science and Engineering in the University of California, San Diego. His work is in the areas of machine learning, computational statistics, information theory and their applications. He is best known for his joint work with Dr. Robert Schapire on the Adaboost algorithm. For this work Freund and Schapire were awarded the 2003 Godel Prize and the 2004 Kanellakis Prize. Freund was elected fellow of AAAI in 2008. Freund is included in the Thompson list of most highly cited scientists: ISIHighlyCited.com . ICML 2009 Tutorial and Workshop Summaries Overview of Tutorials and Workshops As in previous years we were pleased to have a strong program of tutorials for ICML 2009. These were held on June 14, immediately preceding the main conference. The program featured nine tutorials covering a wide range of methods for, and applications of, machine learning. There were tutorials on: active learning (Dasgupta, Langford); convergence of natural dynamics in multi-agent games (Even-Dar, Mirrokni); machine learning for large social and information networks (Leskovec); learning with dependencies between several response variables (Tresp, Yu); machine learning in information retrieval (Bennett, Bilenko, Collins-Thompson); the neuroscience of reinforcement learning (Niv); reductions in machine learning (Beygelzimer, Langford, Zadrozny); structured prediction for natural language processing (Smith); and a survey of boosting from an optimization perspective (Warmuth, Vishwanathan). We would like to thank the community for the high-quality tutorial proposals that were received, the presenters for their extensive efforts in preparing and delivering the selected tutorials, and the local arrangements, program, and general chairs of ICML for their hard work in organizing such a stimulating conference. Jennifer Neville ICML 2009 Tutorial Chair Once again, ICML solicited and hosted world-class workshops on topics related to machine learning. This year, we were delighted to collaborate with the program co-chairs of UAI (Jeff Bilmes and Andrew Ng) and the COLT workshops chair (Sham Kakade) to put together an exciting joint program. We constructed a slate of nine workshops that represent a wide range of perspectives and fields, as seen in the summaries below. All workshops were held on June 18th, immediately after the main conference days. We would like to thank all of the workshop organizers for their service to the community in putting together these high-quality meetings. We also thank the outstanding local arrangement chairs and the general and program chairs for ICML and the other conferences for creating another exciting and successful conference. Chris Williams ICML 2009 Workshop Chair